A note on the optimality of frequency analysis vs. lp-optimization

Naveed, Kamara, and Wright’s recent paper “Inference Attacks on Property-Preserving Encrypted Databases” (ACM-CCS 2015) evaluated four attacks on encrypted databases, such as those based on the design of CryptDB (Popa et al., SOSP 2011). Two of these attacks—frequency analysis and lp-optimization—apply to deterministically encrypted columns when there is a publicly-available auxiliary data set that is “well-correlated” with the ciphertext column. In their experiments, frequency analysis performed at least as well as lp-optimization for p = 1, 2, and 3. We use maximum likelihood estimation to confirm their intuition and show that frequency analysis is an optimal cryptanalytic technique in this scenario. 1 Overview of the attacks and attacker’s capabilities Naveed, Kamara, and Wright evaluated two attacks on deterministically-encrypted database columns when the attacker has access to an auxiliary data set [1]. Frequency analysis decrypts the column by matching the most frequent ciphertext with the most frequent plaintext from the auxiliary data (and so on for the other less frequent ciphertexts). lp-optimization decrypts the column by matching the frequencies of the ciphertexts with the frequencies of the auxiliary plaintexts in a way that minimizes the lp-distance of their histograms. Existing adversarial models do not describe attackers who have access to auxiliary information. The two above attacks are not ciphertext-only, since the adversary also has the auxiliary data set, nor are they known-plaintext, since the adversary does not actually know any plaintext-ciphertext pairs. Instead, they could be called “ciphertext with frequency data” attacks. Decrypting a deterministically-encrypted database column using this type of auxiliary data is analogous to breaking a monoalphabetic substitution cipher given plaintext letter frequencies. However, in the context of an encrypted database column, these “letters” are not ordered, and therefore higher-order frequency statistics (of bigrams, trigrams, etc.) do not apply. Cryptanalysis of monoalphabetic substitution ciphers does not usually consider this case. 2 What is auxiliary data, exactly? The attacker’s challenge is to decrypt a deterministically encrypted column with the help of some auxiliary data. We use the language of statistics to state explicitly what we believe is Naveed, Kamara, and Wright’s assumption: the encrypted column’s underlying plaintext is a collection of independent samples of a random variable that has the distribution defined by the auxiliary data. The auxiliary data is a multiset (i.e., which may contain repetitions) that we write as a vector z = (z1, . . . , znz ), where each zi comes from the plaintext alphabet AM = {m1, . . . ,mn}. Let Z be the